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to whether the search was successful. This paper summarizes the 
information collected on the evaluation forms and identifies at least 
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scale; (3) bibliographic verification of specific works; (4) 
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This paper examines statistical information compiled from reference uses of 
the computer since 1978 in the General Reference Section at Penn Stat6. 
An evaluation form. was completed by the librarian involved each time the 
computer was used to help answer a reference request, and we collected 
such information as: date; status of requestor (faculty, undergraduate, 
etc.); printed sources consulted and time involved; online databases used - 
with time and cost figures; and a statement of whether the search had 
be^n successful. Both* the questions 'asked and the ways -tl\e terminal* was 
used to answer- them varied widely, but upon analysis some patterns have 
appeared.. We have identified at least five major categories of questions 
that are feasible fqr online reference, at* least in a large academic 
library: 1). directory-type information; 2). literature retrieval on a 
limited scale; 3). bibliographic verification of specific works; 4). 
definitions err explanations of concept's or acronyms; 5). book reviews. 
A detailed statistical study concluded that the actual nature of the 
question seemed to determine search success or failure. 
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1. INTRODUCTION 



Computerized literature searching has been , available on a fee basip 
to our clientele at Penn State since 1975. As the librarians' * online^ 
expertise developed, and as more databases in more subject' fields became 
available from the Vendors, we discovered another attractive use for ouf\ 
computer capabilities — to answer various reference questions that could 
nqt be readily 'found in the printed sources available, or that we felt 
could be answered more economically online. This dimension of reference 
service resulted in a steady trickle of terminal uses f<?r ready reference, 
and all five of the current Penn State searching locations (Life Sciences, 
Government Documents, General Reference, Physical Sciences, and Engineering) 
have used the terminal to some extent to answer reference questions. * 

In J<uly 1978 the General Reference Section began to keep a fairly 
detailed record of each attempt to use one or more online databases in 
answering questions for patrons. Our purpose, aside from professional 
curiosity, was to amass enough data to justify our request for a terminal 
at the Reference Desk (w£ presently use the literature searching terminal, 
locate*! in a separate room afr least 50 feet away from the desk). Since 
- General Reference is primarily responsible for reference service to "the 
social sciences and humanities, this study will reflect that emphasis, which 
is particularly apparent when we detail the databases used and printed 
sources consulted . 
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2. RESEARCH METHODOLOGY 

Although much has been written on comparisons of manual vs. computerized 
literature searches, we coyld find very little in the literature to use as 
a guide for developing the reference aspect of computer se^rchefe (1, 2, 3). 
' To begin the study, we listed all the types of information likely to be 
useful in evaluating such a service. One of our main purposes was to 
discover the various ways the computer could be creatively used to aid 
us in our work. At the National Online Meeting in 1980 Jari Egeland of 
BRS discussed a number of tjiose uses "(4): 

-when a term or concept is difficult to locate; 
-citation verification; \ 
^-searching a file^you do not have in hatd copy; 
-use of CROS dor Lockheed's DIALINDEX J to direct patrons to the most 

appropriate reference source; 
--to find subject descriptors or terminology most appropriate to use 

j.n a manual index; h 
-a few good "hits' 1 to get a patron started, and 
-directory information. 
In the 232 uses of the Computer that we have evaluated *so far, all of the 
above possibilities were explored at one time or another, and will be 
examined using the five categories we ourselves identified. 

3. THE EVALUATION FORM 

The evaluation form that the Section began using in July 1978 
asked for the date; searcher's name; status of requestor; the question; 
specific printed sources used, if any, and Dime involved; # online databases 

used: online time and cost; and an evaluation by the Searcher of whethet 

I 

or not the online search could be considered successful. The completed 
evaluation sheet was stapled to the search printojat (patrons could 
photocopy the printout if they wanted to) and filed for future reference. 
After 30 months, July 1978 — December 1980, and 232 documented reference 
uses of the terminal, some patterns have begun to appear. < 

3.1' Status of Requestor 

At Penn State's main campus the Library selves about 38,200 students 
(Fall 1980 figures), the majority of whpm are undergraduates. This group 
is ^traditionally the least likely to make use of our fee-based literature 
^search service, so we were curious to see whether we were reaching some 
of them through our quick reference searching. . Fifty^four of the s'earches, 
or 23%, wer£ for undergraduates', and, not surprisingly, the vast -majority 
of their requests were answered by supplying literature retrieval on a 
limited scale. Of the remaining requests, 34 (15%) were for* teaching 
faculty; 58 (25%) for library fadulty or staff, or other univefsity 
staff; 52 (22%) were for graduate students^l3 (6%) for f 'others f ' (mostly , 
people from the surrounding community or ^focal businesses) ; and 21 (9%) ^ 
were unknown. Since there has never been, within memory dsf the present 
library staff, an analysis of the users of the 'reference trote^ijSit is 
tempting to suspect that similar proportions of patnons may apply to th§ 
harder questions^rece^ved at the desk; this postulation would need further 

research to confirm or deny it. 
n 



4. CATEGORIES OF QUESTIONS t 
» * 

Certainly the most interesting part of our study turned out to be an. 

analysis v of the questions themselves, and the. variety of ways the terminal 

was used to answer them. For example: >* 1 

1). A faculty member wanted to find references to any dissertations 

\ that related to athletic departments and their structures within * 

colleges or universities. The seaSrch through the printed 
dissertation sources had been futile, but wherj a librarian tried 
Comprehensive Dissertation Index online y one excellent disserta- 
tion was found .in just three minutest 

- m 2). A faculty member needed to find the correct bibliographic P 
information for an article by Benjamin DeMott that had appeared 
| in American Psychologist at an unknown date. After giving up in 
frustration with the printed indexes, he asked a librarian for 
help; in thtee minutes online, the correct reference to Psychology 
Tyday was 'found. 

3) . A graduate student needed information on the elderhostel program 

and was having very little luck. Five minutes online produced 
five excellent citations. 

4) . An undergraduate was looking *for information on the motor skills 

needed in*- dart throwing.. After v spending over an hour with the 
printed Psychological Abstracts searching under motor performance, 
she came to a librarian who, at a cost of less than $4.00, found 
exactly the citations needed. \ 

5) ., A librarian needed to find a government document (for a class.on 

the brewing industry) that, she thought was called "Beer Is a ' 
Four-Letter Word." After spending ten minutes with the printed 
Monthly Catalog , she tried the corresponding online database and 
found that the document was called "Is Sefer a Four-Letter Word? 11 
In examining our terminal uses (See Table 1), we found that they 
could be categorized like any other reference questions. The five 
categories we set up reflect the kinds of printed reference sources one 
might use to find the answers: 

I. Directory Type: institutions, nampe, addresses, biographical 
information 

II. 'Literature retrieval on -a 'limited scale* 
III. Bibliographic verification of /specific works: books, dissertations, 
newspaper articles, documents, reports, conference proceedings . 
IV. Dictionary or encyclopedia type: definition of a term or 
acronym; explanation of a /concept 
V. Book reviews 



*Literatur$ retrieval, on a*. limited scale- is fairly distinct ffoin 
fee-based literature searching. The former is initiated by the 
/reference librarian when online access seems to be the logical way 
\o handle a question, and the result is usually up to ten references, 
with perhaps five minutes spent online. /The full-scale literature 
search is requested by the patron and implies a thorough coverage 
of a topic. , • 
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"Table 1: Categories of Questions 



Referring to ^Column H in Table 1, t'he low success rate of directory-' 
type questions (33*%) relates to the use of pointed directory sources. 
Questions at the reference desk requiring directory information are usually 
answered pretty readily by a standard group ot printed sources kept a few 
feet behind the desk. The more difficult questions require us to range 
farther afield, and at this point an online database used as a reference 
source is probably just as -likely to provide an answer as a printed source. 

5. BIBLIOGRAPHIC VERIFICATION 

Bibliographic verification questions, Category III, were ^further 

subdivided by type (proceedings, articles, etc), as shown-in Table 2. 

The single advertisement verification is another good example of succel^- 

ful reference use of^ the terminal. During the week after John Lennon 

was killed, a local minister planning a memorial service came in to 

try to locate a full page ad on peace that Lennon had placed in the 

New York Times sometime in 197 9. Using the New York Times database on.; 

BRS, a librarian found the correct citation in less than three minutes; 

a manual search could have taken hours. 
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Table 2: Question "Types Within 


Category III 




<6.- DATABASES AND PRINTED SOURCES 
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Fifty-three dif f erent^online "databases 


were consulted at 
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once, though only fourteen were used at least 


six different 
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and seem worti^ noting.- (Table 3 5 
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Table 3: Online Databases Used, 



One hundred thirty-six different pointed sources were used at 
least once, but of these only 17 were used six or more times. 
(Table 4) . \ 
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Printed Source 

ERIC (RIE or CUE) 
Psychological Abstracts- 
Comprehensive Dissertation Index or 

Dissertation Abstracts International 
Card % Catalog 

Social Sciences Citation Index 

Reader 1 s Uuide 

Education Index 

Books in Print 

New York Times Index 

Cumulative .Book Index 

National Union Catalog 

Encyclopedia of Associations ( 

Business Periodicals' Index 



t Forthcoming Books 

•Ulrich'-s International Periodicals Directory 



Number of Use& 

45 
39 
22 



22 

20 

18 

18 

13s 

12 

7 
7 
6 

, 6 
6 



Table 4: Printed Sources Used 



We were not surprised to find that four of the top five sources in 
each list directly correspond. Judging by the number of printed book and 
journal title verification sources used, once our reference librarians 
have access to RLIN it will undoubtedly place high in the database ranking 
certainly among the first five. We'will have our opportunity to judge 
when the' RLIN database becomes available to us in the first half Df 1981. 



7. 



TIME AND COST 



All 232 evaluation sheets indicated the amount of online time involv 
in the search, and most estimated the amount of time a librarian had spent 
on manual searching as well. £able 5 indicates the results of this aspect 
of our. study. 
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Aside from librarian time and, online • time , the many other costs of 
dnline searching have 1 been consciously omitted since the terminal was used 
primarily" for literature searches throughout the study periqfl, "and for 
reference retrieval' only * if the terminal was not being used/for a scheduled 
search. (See (6) and (7) for fuller * treatments of the geneyfei question of* 
costs) . 

The mo>st interesting result of the calculations in Tfable 5 was ^hat 
there' appears ta be an insignificant difference in the average cost ~of 
searching them Manually , We would therefore not expect tMe librarians 
to exhaustall printed sources be-fore turning-to an online database; and 
as' with any* reference question, a trained professional should turn to 
whatever sources are ra>_gt likely to answer a query, in jthe most efficient 
way possible. ^ 

Sometimes-the'patron's- time was also noted on thfe form, and allows 
speculation bn the' amount of t^me our readers may waste in fruitless 
searches for information. N Their ' tim^was:; recorded onf many of the 
evaluation sheets, at an, average of ^57 minutes per search. These state- 
ments of patrons 1, time were based on chance comments/ (*L 1, ve been searching 
for ^n hour now, and haven't found anything. n ) ,$^and /are probably not 
precise. - 
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THE FUTURE 



ERLC 



Once a terminal is provided, at the Reference Deak, its.use for N 
answering reference questions will 'increase steadily. \Every month new 
online databases are announced that would have obvio^^ applicability for 
ready reference us'e. , Some correspond to printed reference books, Where 
the online use would offer new forms of access or more uA-to-date 
information, whi'le others have no printed counter part. , As use'of the 
databases lop r^ady reference work continues to develop nationwide, it 
will reach the point where the first reference librarian ori dpty in th£ 
morning will dial-up tht local telecommunication, node (Telenet or Tymnet) 
and. the r^erence desk -terminal will stay connected all day. The librarians 
will thus' be able tp quitkly choose different vendors and databases that 
they need to use on a moment by moment basis. Needless to say, lengthy 
and time-consuming evaluation sheets" will be Replaced" by statistics 
gathering by sampling methods. 

Other than the growth of this serviced, which seems so obvious and 
inevitable,, the nature of the reference .questions and v the manner in which 
librarians handle them is likely to change afto.° The above discussion of 
five different categories of reference questions which wefe handled online 
is based unavoidably on the printed-ref erence-book mode of thinking. While 
librarians commonly categorize questions into directory type, dictionary 
type, encyclopedia type and so forth, it is really the category of the 
reference book used to answer 'the question that is applied to the question 
itself. The terms commonly used to/categorize online databases — biblio- 
graphic and non-bibliographic — hardly fit the need for evaluating and 
understanding the online re'ady reference questions. 

Thus, a clear description of the categories of questions which* may 
be answerable oriline is certainly not possible at this time. New types 
of reference de^k services — such as interactive manipulation of statistical 
databases — -could represent categories, of reference questions which are 
not at all part of present reference desk work. The only conclusions 
possibie are that online ready reference service is certain to. grow and to 
j*hange> significantly the nature of reference service as we know it today. 
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